From 1907 to 1915, Sergei Mikhailovich Prokudin-Gorskii, a Russian chemist and photographer, travelled around Russia documenting the Russian Empire with colour photography. He was able to do this on black and white film, by taking three exposures of the same scene with red, green, and blue filters. The three photographs can then can be projected through red, green, and blue filters which superimposes into a single, fully coloured photograph when aligned correctly.
The glass negatives provided by Library of Congress. The three images correspond to the blue, green, and red filters, respectively. At a high level, our program needs to extract the three color channel images, place them on top of each other, and align them so that they form a single color image. We can then apply additional image processing techniques to produce a vibrant but realistic image with as few visual artifacts as possible. We will go through each of these steps in detail.
To create colored images, the three channels (Red, Green and Blue) must be aligned correctly. Since the original images are stacked on top of one another and have artifacts, we required to do some preprocessing align images correctly.
This is my step by step approach:
Divide tensor to 3 equal parts and get B, G and R channels. In the original images ordering top to bottom B, G, R
Results of stacking 3 channels without alignment
Images has a artifacts on edges. This creates problems for correct alignment. For a proper alignment I've cropped the borders to get better results
Align the cropped red and green channels to the blue channel to get (x, y) displacements for both channels. There are many algorithms that can be used to achieve this, and will be discussed below. Apply the displacements onto the uncropped versions of their corresponding channels. The channels without their borders cropped is used instead of the cropped versions to preserve as much of the image as possible. Merge the three channels together into one final image and save.
I choose Normalized cross-correlation (NCC) to solve the misalignment issue. The NCC is calculated as:
def ncc(a, b):
#normalized cross-correlation: (a./||a|| dot b./||b||)
return ((a/np.linalg.norm(a)) * (b/np.linalg.norm(b))).ravel().sum()
Result of Normalized cross-correlation alignment
Most of the photos are correctly aligned with NCC algorithm. But we can clearly see photo of the Emir(7th photo) is not aligned correctly. Comparison in colour intenstities has its limitations, and does not work in some cases. Edge detection does a comparison between the changes in intensities instead. There are many approaches to edge detection such as Sobel edge detection and Canny edge detection.
To solve to problem I've used Canny Edge Detection align images
The Canny edge detector is an edge detection operator that uses a multi-stage algorithm to detect a wide range of edges in images. It was developed by John F. Canny in 1986. Canny also produced a computational theory of edge detection explaining why the technique works. Edge detection is also helpful in aligning images that have a lot of colour warping.
Result of edge detection on Emir
Original image has a deformations. I think we can't solve the problem using classical image proccessing techniques. Deep learning can help this kind of situations
Image 3 looks super greenish and don't has a any color depth. Histogram equalization adds its color depth but does not produce a perfect result. If we had a higher quality image we can perform better results
Evolution of Image 3 (No Proccessing/Edge Alignment/After Enhancement)
I think light comes through to left side of the image. This makes a left side more brighter. Histogram equalization solves the problem
Evolution of Image 5 (No Proccessing/Edge Alignment/After Enhancement)
Autocontrast helps Image 6 to get a warmer look and color
Evolution of Image 5 (No Proccessing/Edge Alignment/After Enhancement)
In image of the lady image dont have a depth. Lady is clear but background is overbright. Histogram equalization helps us to see big picture
Evolution of Image 9 (No Proccessing/Edge Alignment/After Enhancement)
Image 15 suffers from the gamma. Grasses are not vivid much. Gamma correction makes small improvement. Now grasses looks vivid
Evolution of Image 15 (No Proccessing/Edge Alignment/After Enhancement)
Problem with the seller is image looks washed and some areas are super dark. Histogram equalization gives some its colors and solves the darkness problem
Evolution of Image 26 (No Proccessing/Edge Alignment/After Enhancement)
Actually Image 29 looks great but it has cold feeling. Looks blueish. Autocontrast rescues the image from this bad feeling
Evolution of Image 29 (No Proccessing/Edge Alignment/After Enhancement)
